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ABSTRACT 

This  work  examined  minimax  linear  estimation  in  multiple 
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studies.  In  particular,  flaws  in  other  simulation  studies  of 
ridge  estimators  were  depicted.  Consequently,  an  improved 
simulation  procedure  was  used.  It  was  observed  from  these 
studies  that,  contrary  to  published  statements,  a  ridge  estimator 
can  be  considerably  superior  to  the  ordinary  least  squares 
estimator,  especially  when  high  pairwise  correlations  exist 
among  the  regression  variables.  Robustness  considerations  were 
used  to  suggest  a  requirement  that  a  "good”  generalized  ridge 
regression  estimator  should  satisfy. 
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ABSTRACT 

This  paper  considers  minirnax  linear  estimation  of  the  para¬ 
meters  in  a  multiple  linear  regression  model.  Recent  results 
are  summarized,  and  some  new  results,  including  a  transformation 
invariance  property  of  minimax  estimation,  are  given.  These 
minimax  estimators  of  the  parameter  vector  can  also  be  classified 
as  ridge  regression  estimators  with  nonstochastic  ridge  para¬ 
meters.  Some  ridge  regression  estimators  with  stochastic  ridge 
parameters  can  be  motivated  by  minimax  estimation  considerations. 
These  minimax-motivated  estimators  are  examined  in  several 
simulation  studies  and  some  observations  are  made  based  on  these 


simulations  and  minirnax  theory. 


1 


INTRODUCTION 


The  usual  multiple  linear  rear  ess  ion  model  is 

-  =  -  -  *  5 

whore  b  is  a  vector  c:  uncorrelated  random  variables  wit;:  ;:.Cc 
zero  and  variance  o“,  and  X  is  a  full  rank  n  *  a  matrix  with 
h  <  The  usual  procedure  for  estimating  1  is  the  least  sqt 

method.  It  is  well  known  that  this  method  is  equivalent  to 
minimum  variance  unbiased  linear  (MV'JL)  estimation.  Simi  Ur  ly 
the  usual  met h.cd  for  estimating  a  given  linear  combination  of 
ino  cool i  id  is  .  e  ^  .  m  recent  \\  urs,  nary  eonlo  r<me 

attempted  to  reduce  tne  near  square o  orior  ( MS K )  by  allowing 
some  bias  in  their  estimators.  One  such,  biased  estimation 
nroceii.ro  is  ridge  regression,  which  was  first  studied  by  Hoc 
and  Ken rare  L  7  1  .  Ricga  regression  estimators  have  the  form 

$*{<)  -  [  -V  *  -V  +  kl f 2 X'Y 

whore  the  ridge  parameter  k  is  either  nonstochastic  (based  or 
prior  informs  t  lorn )  or  stochastic.  It  lolKws  from  a  im.m;  i  i 


3  .  N  t M ii  K I C  A L  K bob’  LTG 

Within  the  past  five  years,  many  ridge  estimators  have  appeare 
m  the  literature  and  ;avo  been  compared  in  subsequent  simulation 
studies.  Golub,  iieuth,  ana  Wanda  L  5  j  state  that  two  dozer,  is 
probably  a  conservative  estimate  of  the  numeer  of  such  es timu tor s . 

Many  practitioners  have  used  the  ridge- trace  approach  to 
select  the  value  of  the  ridge  constant.  This  approach  is  highly 
subjective,  however,  and  has  also  been  criticized  on  other  grounds 

though  more  formal  methods  for  selecting  the  ridge  constant 
need  to  be  employed.  The  ridge  estimator  presented  m  Section 

A  j  /  a  a 

where  is  stochastic  ana  equal  to  has  been  t he  local 

p oir.t  in  our  s i m u  1  a 1 1 o n  stuiios.  S erne  of  t h. e s e  r e v :  1 1 s  a r e 
presentee  later  in  this  section.  »*e  nave  also  stuaiea  hue 
shrinkage  estimator  presented  in  example  2,  but  it  did  not 
fare  as  we 11  as  the  r id g o  e s t i ma tor. 

Before  u n a e r  t a r. i n o  a  simulation  s t u a y  of  i  e  g  i  e  s  s  i  o  n  e s  1 1 m a t o r s 
an  experimenter  must  select  an  appropriate  loss  function ,  and 
choose  a  mechanism  for  generating  data  that  is  represent  a L  ;  vo¬ 
te  f  a  V*  V  1  A.  'n_4  .  i  L  •  1  i  |V.'  i  1  '  ,  P  L  >  .  I  >  l.  k.  1  1  »  .  ■  O  .  *  V--  1  <  k  'ivil'lil.  1  •  >.*  i  A  v«  A  i  ' 

s  q  u  a  l  aj  a  e  t  r  o  i  ,  i  •  e  . 


where  p  eenotes  an  estimator  or  p  (see,  e.cj.  Gibbons  i  4 , 
iier.m'.orle  ana  bran  tie  loJ)  . 

We  have  used  (o)  in  add i Lion  to  the  loss  function 

(i-K)  1  "  '  v 


which:  was  a 


Iso  used  by  Dempster,  Sc hat no f 1  ,  and  Warmuth 
rguad  (as.  they  did)  that  (6)  would  be  an  appr 
cn  if  the  primary  goal  of  a  regression  study 
or  tne  parameters#  wnerees  (7)  woula  a e  atpre 
ession  was  to  be  used  for  prediction  (since  ( 

,  *•  i  ‘  M  ■;  v  —  y  . . 

a .  j  i  *  *  *.  , .  ;  \  •  -  . .  y.‘  j  ,  <a  1 1  a  .  s  •  l  j.  a  f. '  1  Cu  a  ^  a  a 

e  loss  f  unction  v  i  —  a  pJ  *  <  ^  —  a  pJ  woula  se  mappr 
estimation  or  prediction  since  the  loss  is  mi 
i ia  1  e su  s  t  s , i  u a  r  os  a s  t  i me  tor#  a  s  w a  s  mo n  1 1  o : . e d 


f  w e  ado p t  either  ( 6 )  or  ( 7 )  o r  bo  1 1 : ,  w e  t h. e r.  :;iu s  t 

und  p  to  usa  as  well  as  the  method  i or  ganorating 

Ncwhouso  ana  Oman  L10J  sh.owed  that,  assuming  a,  g 
a 

e  fixed,  Kdb '  •£*(<))  is  maxim i  red  #  for  3’t-  i,  when 
rmaliuad  eh;r;;voctor  ror  respond  l  :s  ;  to  the  smallest 


It  can  be  are 

loss  function 

i  f 

estimation  or 

t  i  iC 

if  the  r ogres 

si  on 

be  written  as 

/  vT 

l  *•  fw'  ' 

-  ~  A  ^  -  f!he 

loss 

for  either  es 

ti::-ia 

when  £  is  tha 

lea 

Section  2. 

c  ,  .  : 

jl  x.  w  a  c.  ^ 

Opt  i 

what  X  and' p 

to  u, 

or  1 .  tewhou 

a  e  a 

k  io  be  f i x e a 

,  MG . 

l.  h  a  i.oi  mu  1 1  *■  •  a 

a  e  i  . 

r*  r 


use  these  tv; o  o  ho  ices  f o r  3  .1  n  t  he  ;i  r  s 1 mu 1 a  1 1 o n  : ;  t  uJ  i e s  . 

is  seiu-c t^e  in  suer,  a  way  as  to  make  /.  1 A  an  equicorrciu 
matrix,  and  observations  on  the  dependent  variable  are  th 
generated  as 

y  =  a  3  +  c 

where  u  is  .Yfego“/.  Several  values  of  otJ  are  then  used  i 


’y  pi  cal  ly 


t on 


( 3 ) 


ine  loss  i o i 
( 6 )  or  ;  7 )  ,  c 


junction  wish  each  of  several  X  matrices. 

estimator  is  then  determined  using  ei th.ci 

This  cone ral  pr  oc«. 

we  would  not  expect  to  encounter  an  cquicor rela t ion  matrix  with; 

actual  data.  decora,  although  the  two  ci.o ices  for:  f  cc 

to  the  o e s  t  a r a  to  u r. e  w o r x  t  c a s e  i  e r  r  i  u  j e  r  o ^ j  i  urn  o i .  e s  s  1 u . a  t ■: 

(in  terns  of  mean  squared  error),  we  advise  against  ..sing  the 

smallest  eigenvalue  ease*  the  reason  car'*  e e  uiscirr.ci  1 r  u m 

Table  1.  In  particular,  wu  si'.ould  notice  win t  iug  .-:r:  to  t iv_- 

average  value  of  ;ihe  coefficient  oi  multiple 

as  we  move,  Keeping  s  fixed,  from  t  lie  v;e  1 1  -core  1 1  r  cm  a  ir  l  x 

in  the  first  part  of  the  table,  to  the  highly  i  1 1  -  cone  t  n  neu 

matrix  in  the  bottom  part  of  the  table.  One  can  ooserve  U;s 

2 

in  Table  1)  that  will  be  approximately  q/n  in  the  smallest 


eieen va  i  lie  c v.- s e  e v e n  i. c >  r 


C  O  *  1  Cl  L  v-  i  O  i .  l  1  ..  . .  ^  a 


-Id- 


W  O  al'O 


t hus  unable  to  sco  how  poorly  a  ridge  estimator  por:or;:s 
relative  to  i^ust  secures  lor  reasonable  values  oi  R“.  lince 
progressive  ill-conditioning  tends  to  cause  R"  for  trie  smallest 
eiger.value  case  to  be  much  different  from  R“  for  the  largest 
eigenvalue  case,  making  c~  smaller  so  as  to  produce  higher  Re¬ 


values  for  the  smallest  eigenvalue  case  v;oi 


:ly  1.0  in  the  largest  eigenvalu 


.  ^  uOi.u  l.  ..!a;.L  i\ 


e  lIjU  . 


procedure.  We  have  also  generated  observations  on  i  as  in 


for  each  trial,  a  3  vector  is 


U  'vJ  l  .  U  O  ^  V-A  I  o...  .J.  ^  O  ..  i  . 


distribution  on  the  collection  of  all  norm-one  q-oiement  vectors. 

A  normal  ( 0  #  a k' )  error  vector  a  is  then  generated  and  1  is 
computed  as  i  —  A”  3  t  a.  This  tod  l  lo  j  cautions  against  the  use 
of  random  3  in  simulation  studies  winch  include  a  liferent  classes 
of  estimators.  V.’ e  s e e  n o t  h i n g  w rone,  how e v e r  ,  v,  1 1 : ;  u . ;  i  n ■; ;  3 


vectors  that  are  uniform  on  the  collection  of 


.±  norm-one  vectors 


when  comparing  estimators  within  a  particular  ciu.a;  such 


r i a e e  estimators*  / \ s  will  o e  seen  A a tei 


ii.am.  o o  j  e  c  1 1  v e  s 


of  our  numerical  studies  have  been:  (1)  to  compare  our  ridge 

estimator  with  the  much-referenced  Hcerl,  Kennard,  and  buiawiu 
C3J  estimator,  and  (2)  to  determine  under  what  conditions  our 
ridge  constant  or  a  multiple  of  our  nugo  constant  would  be 
appropriate . 

In  vijo  l  t  ti’,1  t a  r  oil  forest  urecirunrc  i\  r  a..:,  :ro  l  si;  /  , 


wo  have  als^  -enc  in.  tod  A'1  A'  macaco,:;  m  a  differ  era.  manner.  ln:;U 


oc  me  eomeerroiaC io!'.  approacim 


,  see  t. ! . mi'„  Lr. uu  o  j.  ir.  Kvur. 


13  ]  to  goneiaco  several  progressively  ill~co;:eitimm  matrices. 

The  results  of  our  major  simulation  study  are  shown  in 
ible  2.  The  following  can  be  discerned  from.  ir.snccLion  of  the 


.able .  Fo  i 


•  O  GCis  tjC 


c  o  n stunt  a  e  no  r.d  s  a  r  t  r.  o  s  i  a 


i r c e  ~  i ;n pile s  t  he  n u e d  f or  a  site  1 1 


4^1  W  C  O  4  4  4 )  C  L,  4  4  * — ■  t  C>  4  4  L*  V  -4_  V - O 


versa.  Secondly,  for  fixed  A  ,  the  size  ot  tne  aeare. 


ronstunt  var.es  v:itr. 


ULsjreo  of  1 1  j -cor.d  l  t ion lr.c;  mi  t  :n 


greater  ill-cor.di  tior.ir.g  implies  the  need  for  a  large. 


constant . 


follows  1: 


discussion  in  Section  2  ccnccrr.ir.r;  r onus  t 


f  a  nonstoenus t ic  race  constant  <  will  result 


n  the  riden 


r.b^  orOVIGc 


Lonscquer.tiv  / 


.in.ator  outperf  orr.ii :;g  least  sauarer  :  n  term.'  o : 


;here  /: 


hxu.nnl < 


1)  Simula  b 


e  a  Ov. 


^stochastic)  choice  for  < . 


be  motivated,  in  mart,  by  the  results  i n  Table  2, 

A  a  y 

fact  that  1  i\h' /  2  1  2 ;  should  be  less  than  2  ( c  “/ 2  1  iJ 

A  A  ^  -  J 

1 1  v )  Fi'p'SJ  =  £*2  +  o"2  A.  ,  where  A  A  A  a 

./  _  ,  l  1  -l 


(a.,.-  u;m.  no  normal  ■ 


the  ei son- 


values  of  AfA  and  ow  is  an  unbiased  estimator  of  o". 


A  A  A 

One  rings. t  argue  tnat  p*  p  in  li.  should 


i  cni  mM!  m  y 


so  as  Cu  mu r* 


.  <»i ; ■  n- r o x  ;  mate!  v  u n b  i  a s  ^  u  .  V* i e r.  1  /.  is  i: \  c a r r e l  ^  1 1 o r. 


TABLE  2 


e  1  l -eo;:d  ;  t  i  oaed  X  ::i,iLri.\ 


o 

0“ 

C  1 

'0 

> 

Least  Squares 

s' 

4li  1 

i 

‘s;1 

i 

Cli  , 

lUn~ 1 

Loss 

000  i 

.  996 
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.  L  t’0“t  0  2  0 
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1 . 0u05 
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-cond i t io; 

ied  a  r;a 
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2 

2 

Avg .  R4” 

Least  Squares 

(A  -  [ 

"1 

.  A  ~  1 
"*  “  | 

Less 

.001831 

1.0001 
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1 .0024 

1  .0053 

1.2299 

1  .0136 

0001 

.993 

.0004198 
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1.0117 
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.004233 
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1 .0644 
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0 . 9  3  5 
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600  "lioou”  (K“  >  .7)  observations  were  generated  lor  each  matrix  comb  i  na  L  ion .  The 
entries  represent  average  *.uss  ior  least  squares  ar.d  the  ratio  oL  average  loss  Lor  each 
CiiLiCitor  to  the  average  loss  lor  least  squares. 

Loss  A  is  g  ,  8  %i  -  .  7  Loss  L>  is  ' mm  -  .  /  LVLV,  ;■  -  „  7 
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A  ,\ 

Tiie  --.at rises  used  here  are  the  sane  as  those  used  in  TASLX  1  . 
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A  r.inir.'.jx  approach  to  linear  r o g roscicx  war  p:  cror.tcu  ur.a 
sore  of  the  pertinent  mini  ::;ax  results  of  an  imp  or  ton  t  Russian 
paper  by  Kuks  ana  Giman  were  also  a is cur sou.  .*-.uai  Lionel  ;ur.i::;aA 
results  wore  presentee  including  an  invar.ioncnj  property  of  minimax 
linear  regression  estimators.  It.  Section  3  some  c;  i tier  sms  were 
made  of  the  most  common  simulation  procedure  for  ridge  re  gross ; or 
comparisons,  and  these  were  also  illustrated  in  Table  1.  Accordingly 
a  air  terent  proceuuro  invoicing  r  t.  a  o . . «  s  w  as  a  s  o  a  o  pr  Os-  e 
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The  mi ri max  regress i or.  estimators  wits  r lag o  constants  •.  . 

A  _  - 

and  f  A  .  "  were  seen  to  perform  well  in  the  major  simulation  stuuy 
although  lT:,T  nay  be  a  better  choice,  especially  if  A'  1  .V  is  highly 
i  1 1  -  co  nd 1 1 1 o n ed . 
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11:1 1  ill'1*.  I  i:t<  r>  nl  .\ ! S h  i*m  i in; 1 5  *  -s  ami  • »:  :v|;i i .  d  dm *e'  n in 
''1,si!*f  '  -M  ilnat  t'aii  In-  mod  :m-mi  !it  1 1  :■  *  trace  of 
lid'/v  K-tlicifii;  •>.  h  h  ;  hr  “-cr  power”  ,.f  tin*-'*  data 
Uh;i!;.  ;.i‘  ;ii'p!;i\'  that  j:or;:i<'<  .-niiM  n.  ■  r i ’  i:i  ;i;r  r:d_e* 
r:uv\ . 

2.  I  *  «  //-  2 :  /•  *  •  C  c 

■  :i' *< i<  :  >\‘i  *  v  -  *  *  . *i  . ■.  ■  ■».[■  : r : \  i 

Xj  -  Zv  ! . ,  r  Z  XA  uni  *;  A  V  lV::  :!.*■  i;m  <d 
filler  >iinuk;m«‘  Mint  is  MSI  .  op:  mud  m  the  usual  » i: i » •  < ■ 
sense  (i  ti  )i  •]  id  i;i  i  1 1  l‘.C.M  w  n  !  i<  » ;  .i  i  n:i  r  I » . !  u  ■  I  i  *  •  m  of  I  hr 
ebj;.  n values  mI  XX  ms-  Z  Z  of  sin*  j *;i : a ni‘*i it-. 

j  n>r  v;  :nui  ti'.  -  unkuow a  MSI!  * »| n ; ;i ::i  1  ndtv  lit  i- 
thus  umi  invariant.  Why.  liicn.  do  Smi:h  ami  (‘unipheil 
feel  that  tin1  tit  “should  !»»*”  invariant? 

An  estimation  procedure  will  >  ii-id  tits  that  art*  in¬ 
variant  in  this  sens**  if  ami  oniy  if  the  estimate  b  ft  } 
ami  the  estimate  c  of  -j  are  mt *  rv<  ‘.at n!  by  tlm  implicit 
formula  c  -  A  b  tor  every  A.  I  hr  ridne 

solutions  that  optimize  ;  h«  •  obndivr  *•  ri ! » •  ri:i  «>1  my 
earlier  comments  are  nut  impliedly  interrelated.  Ami  1 
bvl  i !;at  no  ‘anxiety”  worth  avoaunj,  i<  i m f >! i« 1  by  this 
property. 

It  t"  wei!  known  that  any  pro.-ninn*  that  yield*  im¬ 
plicitly  ini  errelat  <M  eMimates  cannot  have  certain 
<lrs;raole  proper:  ie-.  Imr  r\am;  !r.  1  iiL.'v  i  I  *  t  T  A 1  <‘<*n- 
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r"?  III,;i: '  '  ale  tie.!'  tep  ill  l  : .  *  ■  njh;  direction”  in  the 
"l  1  nk’\  !4,7’e  t  e.sai.i  .  .  »e|i;i  i(  n  j .  that  ran  he 

mieipik  t  •!  a-  -  in  t  h ■  •  -••ii>e  oj  <  them  iiaiti  and 

Via  *1  I*.i7  1  . 


3.  OBTAINING  Pr.EASS.'GNED  VALUES 


m-  mad  :»  «  -  a;  • -p  eid  an;,  pn-a-dirmM  vector  by 
deuce  «>:  a  diagonal  nail  r:x  A  in  *  ic-ir  eouai  hui  i  J .  J  . 


1 :  "t»>-  eiU"id,i>  imm -raii/.-d  rid.e,-  ■  j. — . i •  in  with 
arbitrarv  oriaii:.  a-  in  Smith  and  ( 'a  :..p! -•11'.-  e,p;;:  i,  m 
io.l  \  li.-  t!  i-m*  can  ind* ■■  d  ■•learn  any  pi<  a  —  iun*  <i 

solution  l»\  eiimee  of  tiie  ni^i'  factors.  1 1 • . c,„.  Jiut. 

instead  «*i  0  *.  ■'  1.  one  mm.e  have  t<>  l;o  to  tie* 

extreme  «>i  ihiiiji  ride,**  lac!  • us  •er-'aier  '.nan  urn*  or  *vrii 
nejjaiiv«  .  It  one  is  monitoring  samjile  information  on  the 
varianee-bia^  trad'-**!;.  th-Mi  w ..aid  that  MSM 

is  not  reduecd  b\  >ih*ii  an  exnviii,-  taeiie. 
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Comment 


LAWRENCE  C.  PEELE  and  THOMAS  P.  RYAN* 


I  lie  authors  have  i  otu  :u -nt  «.l  some  interesting 
eritieistiis  but  I  heir  l  ohit  tui-'iis  seem  not  to  form  a  basis 
It »r  eeiu  rai  eomlriiiN.it ion  of  rid^e  t egression.  Smith  and 

I  iiwri  iirc  C.V  Prrlc  A>-ist:ust  !’i‘"!f"ur  la  tl  r  I  )r}i.ii tiuciit  u! 
Mat  i.ctaat  ica!  >i’irt;if-.  <  Vd  I  *■»!;». :.,<m  l  hivrT'itv.  Ntitfolk,  \  A 
J1 'Os  'll.'iina-  tV  Ih.iii  a  \'>i-taht  l,:,"!',""r  m  l!ic  |)r| ..art tn*‘n!  "f 
Quantitative  Analw-,  i  rav*-i>,tv  n|‘  <  ’mrmnaf i,  l 'inciniiat t.  Oil 
I.VJJI  I  ai  *’li  was  >  t  c  >pi  ift  o  i  in  J»aif  Ia  the  An  laiiiv  Oili  r  nl 
Sncimiu*  lu’r-.’iUvii,  L'.mMact  N«>.  AloSH  Pl'Mijn  7n  oiJA. 


Campbell  stale  that  lhe\  ailvoeate  the  use  nj  prior 
information,  but  arc  urn-. unsuitable  widi  e\a»'t  i;m  e.r 
eoiist  rail  its.  Tliey  also  state  that  '‘when  tin-  least  sijuaies 
estimates  are  impret  <,-e.  auxiiiavv  tn'.up.uathtn  is  tpute 
useful.  IVeitdoinformat ion  is  ,,f  dubious  value.” 
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We  nnivi*  that  exact  lunar  const r;um>  :iiv  “dis- 
ci**iifurt  111^”  M-i'ininJy  unrealistic,  as  well)  ami, 

consequently,  prefer  coiim  mints  of  tin*  form  j'rf  <  A, 
or  more  generally  d'/’d  <  A*/-,  when*  T  is  a  positive 
detiiut"  matrix  ami  /■  is  a  positive  constant,  There  is  no 
boundedness  assumption  t»n  d \i  inherent  in  i In*  least 
squares  procedure,  ami.  in  pram  ire,  one  might  set  an  a 
priori  upper  bound  on  iva>.  mat  »!e  values  of  d  ,d-  The  decree 
of  superiority  of  a  biased  estimator  over  least  squares 
will  naturally  depend  mi  how  sharp  a  bound  on  dd  an 
experimenter  can  impose.  Suppose  that  bused  on  prior 
information  only,  one  believes  that  <i  *d'd  i>  approxi¬ 
mately  equal  to  a  number  t  *.  Kuks  ami  t  Oman  197J) 
•showed  that  if  i*  >  o  *d'd  then  the  ridge  regression 
estimator  u  r  n  ridge  parameter  A*  ~  1  )  *  will  out  perform 
the  least  squares  estimator  in  terms  of  mean  squared 
error.  Consequently,  if  fairly  accurate  prior  intormaiion 
about  is  available,  thi<  ridge  estimator  is  prefer¬ 

able  to  the  least  squares  estimator,  backing  prior  in- 
formation,  we  can  still  obtain,  by  estimating  ir'.d'd  troin 
sample  data,  an  estimator  that  tends  to  be  better  than 
least  squares.  It  follows  from  this  line- ot  reasoning  that 
estimators  with  this  underlying  motivation  will  not  have 
“loose  theoretical  underpinnings.” 

Smith  and  Campbeii  note  that  ridge  regression 
estimators  are  not  invariant  under  model  iranMorma- 
tioiis  of  the  furni 

V  =■  (,  A  .1 ) :  A  '  yd »  -r*  «  -  -  t  .  t,ll 

lYele  and  Ryan  (ltiT'.lj  discu.-s  minimax  linear  estimation 
based  on  prior  inn -rmat mn  of  the  form  d  7’d  *j :A, 
which  includes  ordinary  and  generalized  ridge  regression 
estimation.  The  author-  observe  tltat  the  resulting 
estimators 

.iv-.-rS-  -  .v.v]  Yr 

I  t 


are  mvanaui  under  model  1 1  ahM.*;  mat  mas  ot  tin* 
Iwl  lll  l 

In  his  coni  nlnu  i-n  to  i  it  i-  dM  u-Mon.  Van  \A.-iiand 
bricilv  mentioned  specihe  ieontlic;  mg i  re-ult-  iron: 
several  simulation  Mudie-.  li  R  not  surprbing  that  the 
results  can  ditfer  greatly  lrom  one  Mwdy  to  the  next 
since  no  “standard"  .simulation  proeetlure  i>  being  u-*d 
III  point  ot  tart,  it  >et'Iii'.  that  liio-e  « ,f  ||s  who  have 
performed  these  studies  have  fallen  o*  a  number  of 
pitfalls.  In  a  regression  simulation  study,  mie  should  u-e 
appropriate  values  of  the  error  variance  a1.  The  me  ot 
correlation  form  implies  that  one  should  choose  a:  el/.-.- 
to  zero  in  order  to  gem-rate  data  with  r»  a-  /nabi*-  //- 
values.  Cnfon unat  ■! > ■.  lr  i-  quite  sensitive  to  -mai, 
change-  in  consequent iv.  groat  care  jisum  be  e\nvi-<*d 
in  order  t * »  avoid  producing  small  li  vain*--.  the 

usual  procedure  of  letting  d  be  ri  pre-.-mt  d  by  tin*  eig.*n- 
vectors  corresponding  to  the  largest  and  smallest  eigen¬ 
values  of  .V'.V  i>  object i< »nable.  In  the -mall'  X  eigenvalue 
case,  ti1  will  be  approximately  p  n  v.  inn  till*  >malle-t 
eigenvalue  is  quit**  '•mall  and  a1  is  at  lea"*  om*  i.rd*T  of 
magnitude  larger.  Tints,  with  />  -  1  and  t>  100  a 
typical  choie*q  we  could  hardly  -ay  that  w>-  have  gener¬ 
ated  representative  ngre.-don  data  when  Ii:  i-  approxi¬ 
mately  01.  Making  <7 *  -mailer  in  an  eitori  in  reim  d\ 
the  problem  would  only  cau"«*  Ji:  0*  lie  aln:-.-i  exactly 
1.0  in  tlte  largest  eigenvalue  *  a->  .  'There  i-  <*!*  :iri \  a  need 
for  a  better  apprna.  ii. 
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T"  i)  intrit'frii  v 


H.  D.  VINOD* 


Comment 


Certain  p t i ■  - r  l.tiowl*  dge  about  intrm-ic  measurement 
•  rror-  •  :iu  I o •  -  T  veriy  ua  « .rpot ate.{  m  ri.ige  regre">ic»n 
by  adding  ;<  li  ittious  ol.-ervatioti-  to  the  data.  Cmi-ider 
a  standai di/.t  d  regies-;.. n  model  >■  -  Ad  t  u,  where 

.V'A  is  fin*  correlation  matrix  ba^rd  on  n  observations 
among  p  regr*-""ors  and  A'v  is  a  victor  of  correlation 
cuetlici«*;its  with  the  dependent  variable.  There  are 


*  l(  1»  \’iit*i*!  e  ."i;|*rrvi‘i.r ■,  i  'nuinniH*  Stmlu*-,  l  irnimiMir  Analvvis 
s.  rt:>i[i.  Aii.rrtcaa  Trlfplmiu*  Telr^rsiph  a v, 

NJ  uss* \ 


measurement  errors  iu  eaeh  ifgres.-or.  >.»  that  ihe  avail¬ 
able  data  an*  iiuli-: iuguisha!»h-  from  ot  le  r  data  w  here  one 
may  add  a  number  Im-iwo-ii  —  .d  to  .100  b.yond  tin* 
lii"t  published  \ligit  There  may  be  a  similar  error  iu  eaeh 
mean  prj  and  Mandat'd  deviation  iSD.h  The  largest 
measurement  error  in  each  staml.u.lizcd  regressor 

u„  —  .iuH"  —  1)  dSD.rh  where  it  -  b  2 . n,  IS 
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